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ABSTRACT 

Current  numerical  algorithms  for  constructing 
optimal  quantizers  encounter  difficulties  if  either 
an  optimal  solution  does  not  exist,  or  if  local 
minima  are  present.  In  this  paper  we  show  that  an 
optimal  quantizer  may  not  always  exist,  and  present 
a  condition  on  the  distortion  measure,  and  another 
condition  on  the  probability  distribution,  either 
of  which  guarantees  existence  of  an  optimal  solu¬ 
tion.  Then  we  comment  on  the  role  of  symmetry  in 
optimal  quantization  and  discuss  the  validity  of 
one  method  that  has  been  used  to  prove  uniqueness 
of  the  optimal  solution. 

I.  Introduction  and  Prel iminaries 

In  recent  years  much  work  has  been  done  to 
investigate  design  algorithms  for  optimal  quanti¬ 
zation  u-5].  A  popular  approach  is  to  use  numer¬ 
ical  techniques  to  minimize  a  mean  distortion  func¬ 
tion.  With  this  philosophy,  one  way  to  generate 
an  optimal  design  is  to  successively  improve  on  a 
sequence  of  sub-optimal  designs  using  conditions 
which  are  necessary  (but  not  sufficient)  for  a 
stationary  point.  Max  [1]  in  1960  developed  a  set 
of  necessary  conditions  applicable  to  differentia¬ 
ble  cost  functions,  and  suggested  one  algorithm 
for  the  mean  square  distortion  measure.  Variants 
of  this  algorithm  have  been  successfully  applied 
to  scalar  quantization  problems  [1,4].  Linde,  et 
al .  [2],  generalizing  on  the  work  of  Lloyd  [10], 
presented  a  set  of  necessary  conditions  similar  to 
those  of  Max,  but  having  much  wider  applicability. 
Their  algorithm  handles  arbitrary  multi-dimensional 
distributions  (i.e.,  vector  quantizers)  and  very 
general  distortion  measures.  Two  considerations 
arise  in  the  use  of  either  of  the  above  algorithms. 
The  first  question  is  the  existence  of  an  optimum 
quantizer  (i.e.,  a  solution  to  the  optimization 
problem).  This  is  directly  related  to  the  conver¬ 
gence  of  the  two  algorithms,  because  if  an  optimum 
does  not  exist,  then  the  algorithms  either  do  not 
converge,  or  may  converge  to  a  non-minimum  solution. 
We  will  present  an  example  of  the  latter  possibil¬ 
ity.  It  is  of  interest,  therefore,  to  set  down 
some  conditions  under  which  a  global  minimum  may 
exist.  For  twice-dlfferentiable  probability  densi¬ 
ty  functions  an  easily  checked  convexity  test  is 
available  [6].  Essentially  this  determines  when 
the  optimization  problem  has  a  unique  stationary 
point.  However,  there  are  many  probability  densi¬ 
ties  of  practical  Interest  for  which  the  criterion 
does  not  apply.  A  different  viewpoint  was  taken  In 
[7,8]  which  presented  sufficient  conditions  on  the 
form  of  the  distortion  function.  Both  of  these  re¬ 
sults  are  summarized  in  the  development. 


The  second  question  relates  to  the  nature  of 
the  solutions  obtained  by  the  above  algorithms.  It 
has  been  demonstrated  that  these  algorithms  can 
sometimes  converge  to  a  quantizer  which  is  locally 
optimum,  but  does  not  have  the  globally  minimum  dis¬ 
tortion  [2].  This  can  obviously  present  some  diffi¬ 
culties  in  problems  with  several  minima.  Some  modi¬ 
fications  in  the  optimization  algorithms  to  over¬ 
come  this  problem  have  been  suggested  [2,9],  but 
this  greatly  increases  the  computation  time.  A  cri¬ 
terion  to  determine  the  uniqueness  of  stationary 
points  would  be  useful.  For  twice-differentiable 
density  functions  the  existence  criterion  in  [6]  is 
applicable.  In  general,  however,  determining 
uniqueness  seems  to  be  a  more  difficult  problem  than 
the  first.  The  authors  are  not  aware  of  any  other 
results  which  address  the  problem  of  uniqueness. 

Recent  algorithms  already  are  capable  of  hand¬ 
ling  general  distributions  and  complex  cost  func¬ 
tions  [2]  and  it  will  be  correspondingly  more  dif¬ 
ficult  tc  answer  the  two  questions  presented  above. 
Many  of  the  previously  mentioned  results  may  not  be 
applicable  anymore.  Therefore  these  questions  need 
to  be  examined  further  so  that  existing  optimization 
procedures  may  be  used  with  confidence  that  the  re¬ 
sulting  solution  is  indeed  optimal. 

In  this  paper,  we  first  develop  necessary  and 
sufficient  conditions  for  the  existence  of  multi¬ 
dimensional  quantizers.  Then  we  turn  to  the  ques¬ 
tion  of  uniqueness  and,  through  several  examples, 
point  out  some  interesting  results. 

First  we  give  a  few  definitions  and  establish 
some  notation.  An  N-level  k-dimensional  vector 

k  K 

quantizer  is  a  mapping  Q:  R  +  R  which  assigns  to 
the  input  vector  x  an  output  vector  Q ( x )  chosen  from 
a  finite  set  of  N  distinct  vectors  (yi :  y^ e  Rk  , 

i=l ,2, . . . ,N] .  When  optimal  quantizers  are  being 
considered  with  an  error  based  on  the  Euclidean 
norm,  there  is  no  loss  of  generality  in  assuming  the 
"nearest  neighbor"  assignment  rule:  Q(x)  is  that 
member  of  the  set  which  is  nearest  to  x  in  Euclid¬ 
ean  norm,  with  ties  being  broken  in  some  pre-assign- 
ed  manner.  This  rule  will  be  adopted  throughout  the 
rest  of  this  paper.  Scalar  quantization  (k=l)  is 
considered  to  be  a  special  case  of  vector  quantiza- 

k 

tion.  Let  X  be  a  random  vector  taking  values  in  R 
and  having  a  cumulative  distribution  function  F.  A 
measure  of  the  performance  of  a  quantizer  Q  applied 
to  the  random  vector  X  Is  the  mean  distortion  func¬ 
tion 

D  »  D(Q.F)  *  /cQ(||x  -  Q(x) ||)dF(x) .  0) 

Here  ||  - 1|  denotes  the  Euclidean  norm  In  R  and  CQ(t) 
is  an  appropriate  cost  function.  For  example,  the 
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rth-power  cost  function  CQ(t)  -  tr  is  a  popular 
choice.  For  simplicity  we  write  C0fl|x||)  *  C(x). 

Because  of  nearest  neighbor  assignment,  a  quantizer 
is  fully  specified  by  its  set  of  output  levels 
{ y .} .  Thus  when  F  is  fixed  we  may  write  D(Q,F)  = 

D(y^ ,y2*. .« ,yN).  The  quantizer  optimization  prob¬ 
lem  is  to  find  the  global  minimum  point  of  D(y1*y2. 
...,yN),  if  any.  A  quantizer  is  said  to  be  locally 

optimum  if  its  output  levels  form  a  locally  optimum 
point  of  D(y1,y2,...,yN). 

Let  Qn  and  Q  be  vector  quantizers.  We  say 
that  the  sequence  {Qn>  converges  to  Q  if  Qn(x)+Q(x) 

at  all  continuity  points  of  Q.  Equivalently,  in 
view  of  nearest  neighbor  assignment,  (Qn)  converges 

to  Q  if  the  output  levels  of  the  Qn's  converge  to 

the  output  levels  of  Q.  Notice  that  this  defini¬ 
tion  allows  the  limiting  quantizer  Q  to  have  fewer 
levels  than  the  quantizers 

Let  Fn  and  F  be  k-dimensional  probability  dis¬ 
tribution  functions.  Recall  that  the  sequence  (Fn> 
is  said  to  converge  weakly  to  F  (written  F^F)  if 

Fn(x)+F(x)  at  every  continuity  point  x  of  F. 
n  k 

A  function  C:  K  -►  R  is  said  to  be  lower 
semi -continuous  if 

lim  inf  C(y)f^C(x)  all  x. 

6-0  0<||y-x||<6 

An  equivalent  condition  is  that  for  any  { xn>  and  x, 
xn-  x  implies 

lim  inf  C(xn)  >_  C(x). 

It 

A  continuous  function  g:  H  -H  is  said  to  be  uni¬ 
formly  integrable  with  respect  to  the  distributions 
(Fn)  if  ri9(x)|dFn(x)  <  00  for  all  n  and 

lim  sup  f  |g(x)|d F  (x)  «  0. 
a-»  n  J||x||>a 

II.  Existence  of  Optimal  Quantizers 

In  this  section  we  review  some  existence  the¬ 
orems  for  globally  optimal  quantizers,  and  then 
prove  some  stronger  results.  The  first  two  theo¬ 
rems  below  are  proved  in  [7,8].  The  present 
statement  of  Theorem  1  is  a  slight  but  straight¬ 
forward  extension  of  that  proven  In  [8]. 

Theorem  1.  Suppose  that  the  cost  function  CQ( t)  is 

non-negative,  non-decreasing  for  t  >.  0,  and  lower 
semi -continuous.  Then  for  any  distribution  F  the 
distortion  D(y^ ,y2»... ,yN)  is  a  lower  semi -contin¬ 
uous  function  of  the  output  levels.  If  CQ(t)  is 
continuous,  then  D(y1 ,y2». *yN)  is  continuous. 
Moreover,  if  F  is  continuous,  then  D(y^ ,y2,.. . ,yN) 

1$  continuous,  even  If  C  (t)  is  not  required  to  be 
semi -continuous.  0 

Theorem  2.  Let  F  be  a  given  k-dlmensional  distri¬ 
bution  function.  If  C0(t)  is  non-negative,  non¬ 


decreasing  for  t  ^  0  and  lower  semi -continuous,  then 
a  globally  optimum  N-level  quantizer  exists  for  any 
positive  integer  N. 

This  theorem  gives  a  sufficient  condition  on 
the  form  of  the  cost  function  CQ(t)  so  that  (1)  may 

achieve  its  minimum  possible  value.  The  distribu¬ 
tion  F  is  completely  general,  and  all  of  the  assump¬ 
tions  are  directed  towards  the  cost  function.  The 
key  assumption  here  is  lower  semi-continuity;  the 
other  two  conditions  merely  serve  to  define  the  in¬ 
tuitive  notion  of  a  difference-based  cost  function. 
These  conditions  may  not  be  weakened  without  adding 
other  qualifications,  as  the  following  example 
shows.  Let  the  cost  function  be  defined  as 

(  °  0  <  1 1|  <  2 

Co(t)  =  |  1  2  <  |t|  .  (2) 

Now  let  X  take  on  a  discrete  distribution  such  that 
P(X=2)  =  P(X=  -2)  =  1/4 

P(X=2-l/k)  =  P(X=  -2+1 /k)  =  l/2k+2  k=l ,2,... 

(3) 

It  is  easy  to  see  that  with  N=1  level,  the  distor¬ 
tion  (1)  cannot  be  minimized.  In  fact,  any  reason¬ 
able  numerical  technique  for  optimizing  (1)  will 
generate  a  sequence  of  quantizers  with  improving 
distortions,  but  the  limiting  quantizer  exhibits 
worst-case  performance.  Note  that  we  are  concerned 
here  with  global  minima,  and  not  merely  local  min¬ 
ima  or  stationary  points,  of  which  this  example  has 
infinitely  many. 

This  example  can  be  generalized.  Suppose  that 
C  (t)  is  non-negative,  non-decreasing  for  t  >_  0,  and 

is  not  lower  semi -continuous  at  t  >  0.  Then  the 
discontinuity  (of  the  first  kind)  at  tQ  must  have 
the  same  character  as  that  in  (2),  namely,  C0(tQ-) 

<  C0(tQ),  and  it  will  be  possible  to  construct  a 
distribution  with  atoms  at  +t  ,  such  that  the  mean 

distortion  (2)  cannot  have  a  minimum.  Thus  if  we 
wish  to  avoid  this  situation,  we  must  require  CQ(t) 

to  be  lower  semi -continuous.  On  the  other  hand,  it 
has  been  shown  in  Theorem  2  that  lower  semi-contin¬ 
uity  is  sufficient  to  guarantee  the  existence  of  a 
global  minimum.  Therefore  we  have  the  following 
theorem. 

Theorem  3.  Suppose  that  CQ(t)  is  non-negative  and 

non-decreasing  for  t  ^  0.  Then  a  necessary  and 
sufficient  condition  that,  for  any  N  and  any  dis¬ 
tribution  function,  a  minimum  distortion  N-level 
quantizer  exists,  is  that  CQ(t)  be  lower  semi-con¬ 
tinuous. 

In  some  situations  there  might  be  some  uncer¬ 
tainty  about  the  distribution  F  being  quantized. 

This  theorem  says  that  if  we  choose  a  lower  semi- 
continuous  cost  function,  then  no  matter  what  dis¬ 
tribution  might  be  Involved,  we  can  be  sure  that  the 
distortion  achieves  a  minimum.  Moreover,  this  as¬ 
surance  vanishes  if  the  function  is  not  lower  semi- 
continuous.  All  of  the  popular  cost  functions,  such 

as  the  rth-power  cost  function,  satisfy  the  require¬ 
ments  of  Theorem  3. 

Theorem  3  characterizes  a  class  of  cost  func¬ 
tions  so  that  (2)  can  be  minimized  universally. 
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The  converse  question  is  also  of  some  interest. 

That  is,  we  wish  to  characterize  a  class  of  distri¬ 
butions  for  which  the  same  property  holds.  A 
partial  answer  may  be  found  in  Theorem  1.  For  if 
the  distribution  function  F  is  continuous,  then 
D(y1 ,y2»- • • »yN)  Will  be  a  continuous  function  of 

the  output  levels.  Using  the  same  methods  in  [8] 
that  were  used  to  prove  Theorem  2,  it  can  be  shown 
that  the  mean  distortion  is  minimized  by  some  N- 
level  quantizer.  This  gives  the  following  suffi¬ 
ciency  condition. 

Theorem  4.  Suppose  that  the  distribution  F  of  the 
random  vector  being  quantized  is  continuous.  Then 
for  any  cost  function  CQ(t)  that  is  non-negative 

and  non-decreasing  for  t  >_  0,  a  globally  minimum 
distortion  N-levei  quantizer  exists  for  every  N. 

For  example,  the  well-known  Cantor  distribu¬ 
tion  [11]  is  continuous,  although  singular  with 
respect  to  Lebesgue  measure.  Nevertheless,  any 
reasonable  cost  function  yields  a  distortion  func¬ 
tion  that  can  be  minimized.  Theorem  4  is  satisfy¬ 
ing  because  in  many  practical  situations  the  random 
vector  being  quantized  has  an  absolutely  continu¬ 
ous  distribution,  and  so  a  large  class  of  differ¬ 
ence-based  cost  functions  is  available. 

III.  Uniqueness  and  Symmetry  of  Quantizers 


The  results  of  the  previous  section  are  valid 
for  both  vector  and  scalar  quantizers.  In  this 
section,  we  shall  be  mostly  concerned  with  scalar 
quantizers.  Through  examples,  we  shall  point  out 
some  interesting  aspects  about  symmetry  and 
uniqueness  of  optimal  quantizers. 

Many  popular  univariate  density  functions  pos¬ 
sess  even  symmetry,  e.g.,  the  zero  mean  Gaussian 
density,  the  Laplace  density  and  the  generalized 
Gaussian  densities  [12].  When  this  symmetry  is 
present,  it  is  a  common  practice  to  look  for  an 
optimum  symmetric  scalar  quantizer,  i.e.,  one  whose 
output  levels  are  symmetrically  disposed  about  the 
origin  [4,5].  Since  in  this  case  only  half  of  the 
output  levels  have  to  be  computed,  considerable 
computational  savings  are  possible,  especially  when 
the  number  of  levels  N  is  large.  Unfortunately,  it 
is  not  true  that  the  optimal  quantizer  for  a  sym¬ 
metric  density  function  is  itself  symmetric.  So 
this  procedure  can  result  in  a  sub-optimal  design. 
Consider  the  density  given  by 


-lxl/3  +  5/12 


i  xj  <  1 


f ( x)  -  <  7  -  1  x|  /72  1  1  |x|  <  7 

(O  7  <_  |  x|  .  (4) 

This  density  has  two  minimum  mean  squared  two-level 
quantizers  Q(x)  and  Q(-x)  given  by 


Q(x)  ■ 


These  have  mean  squared  errors  D  3  2.61.  By  com¬ 
parison  the  best  symmetric  two-level  quantizer  for 
f(x)  has  output  levels  at  +  61/36  and  mean  squared 
error  0  *  2.74.  Similar  examples  can  be  construc¬ 
ted  for  any  nunfcer  of  levels  N  and  for  other  cost 
functions.  This  example  makes  It  clear  that  we 
cannot  in  general  hope  to  simplify  the  problem  of 
constructing  optimal  quantizers  by  appealing  to 


symmetry.  In  applications,  usually  the  probability 
density  to  be  quantized  is  not  precisely  known. 
Typically,  the  greatest  uncertainty  is  associated 
with  the  tails  of  the  density,  while  the  central 
part  may  be  well  estimated.  If  the  nominal  density 
fQ  is  symmetric,  the  actual  distribution  may  be  any 

member  of  the  mixture  class  of  symmetric  densities 

{ ( 1  -e  ) f  +  eh:  h(x)  is  a  symmetric  univariate 
density! 

for  some  positive  e.  We  show  that  with  an  r^-power 
cost  function,  there  is  some  member  of  this  mixture 
class  of  symmetric  densities  which  has  a  non-sym- 
metric  optimal  quantizer.  First  we  state  a  theorem 
from  [9]  which  will  be  used  in  the  subsequent  proof. 
This  theorem  holds  for  vector  quantizers. 

Theorem  5.  Assume  that  C0(t)  is  a  non-constant 
function  which  is  non-negative,  non-decreasing  for 
t  0  and  continuous.  Suppose  that  F^F,  and  that 
Qn  is  an  optimal  N-level  quantizer  for  Fn.  If  C(x-y) 
is  uniformly  integrable  with  respect  to  ( F^}  for 

each  y  e  ,  then  every  limiting  quantizer  Q  of  the 
sequence  {Qn>  is  optimal  for  F. 

Now  we  can  state  and  prove  a  theorem  for  scalar 
quantizers. 

Theorem  6.  Let  the  cost  function  be  given  by  CQ(t) 

=  tr  for  some  positive  r.  Let  N  be  a  positive  even 
integer  and  let  f  be  a  given  symnetric  density 

O  J.L. 

function  with  finite  absolute  moment.  For  any 
given  e,  0  <  e  <  1,  there  exists  a  symmetric  density 
function  h(x)  such  that  for  the  mixture  density 
( 1  '“£ ) f Q  +  eh,  no  optimal  N-level  scalar  quantizer 

is  symmetric. 

Proof:  For  simplicity  we  take  N=2.  The  proof 
generalizes  easily  to  higher  even  N.  Define 

(  (1x1  -  n+1  )/2  n-1  <  Jxl  <  n 


hn(x)  =  /  -(lx;  -  n-1  )/2 


n  |  x  |  <  n+1 

elsewhere. 


This  probability  density  function  consists  of  two 
triangular  masses  centered  at  +n.  Now  form  the 
mixture  density 

fn(x)  =  (l-e)nf0(nx)  +  enhn(nx). 

As  n  becomes  large,  the  contribution  of  f  becomes 
probability  mass  around  the  origin,  and  that  of  hn 

becomes  two  probability  masses  equally  distributed 
around  +1.  In  other  words,  the  distribution  rep¬ 
resented  by  f  converges  weakly  to  a  distribution 

with  the  cdf: 


c/2  -1  <  x  <  0 

l-e/2  C  <  x  <  1 


It  is  a  simple  matter  to  verify  that  C(x-y)  5 
| x-y| r  is  uniformly  integrable  with  respect  to 
{F^},  where  Fn  is  the  distribution  function  of  f  . 
Thus  Theorem  5  can  be  applied.  Taking  (Qn)  to  be 

an  arbitrarily  chosen  sequence  of  optimum  2-level 
quantizers  for  { Fn> ,  one  sees  that  every  convergent 

subsequence  of  {Qn)  must  approach  an  optimum  con¬ 
figuration  of  levels  for  F.  It  is  easily  seen  that 
there  are  exactly  two  optimal  2-level  quantizers 
for  F.  For  example,  with  a  mean  square  error  cri¬ 
terion  (r=2)  the  optimal  sets  of  output  levels  are 
{-c/(2-e),  1}  and  {-1,  e/(2-e)},  which  are  clearly 
non-symmetric,  especially  for  small  e. 

Since  the  limit  quantizers  are  non-symmetric, 
there  must  be  infinitely  many  densities  f  having 

non-symmetric  optimum  quantizers  Qn-  The  optimal 
quantizer  for  ( 1 -e ) f 0 ( x )  +  eh(x)  is  obtained  by 
scaling  the  output  levels  of  Qn  by  the  factor  n. 
Thus  there  are  infinitely  many  hn  satisfying  the 

requirements  of  the  theore1 

The  contaminating  density  hn( x )  need  not  have 

the  triangular  shape  in  (5).  It  could  just  as 
easily  have  been  made  infinitely  differentiable. 

For  example,  two  Gaussian  bell -shaped  curves  cen¬ 
tered  at  +n  would  have  given  the  same  result.  The 
phenomenon  described  in  the  theorem  has  been  noted 
previously  [7].  However,  the  present  version  is 
more  general  than  the  earlier  one. 

Having  seen  these  examples,  we  might  be  in¬ 
terested  in  finding  some  condition  so  that  a  sym¬ 
metric  probability  density  will  have  a  symmetric 
optimum  quantizer.  To  the  authors'  knowledge,  the 
only  condition  which  guarantees  this  is  uniqueness 
of  the  optimum  quantizer.  For  if  a  density  has  a 
single  locally  optimum  quantizer,  then  that  quanti¬ 
zer  must  be  symmetric.  Fleischer  [6]  derived  a 
simple  criterion  that  guarantees  uniqueness  of  the 
stationary  po’nt  in  the  case  of  mean  square  dis¬ 
tortion. 

Theorem  7.  Suppose  that  a  twice-dif*erentiab1e 
positive  univariate  density  function  f  with  finite 
second  moment  satisfies  the  inequality 

h2 

— «  in  f(x)  <  0  (6) 

dx^ 

for  all  x.  Then  the  mean  squared  error  Ofy-j^,..., 
yN)  of  an  N-level  quantizer  (N  arbitrary)  has  a 

unique  stationary  point,  and  this  stationary  point 
is  a  global  minimum. 

Many  densities  of  practical  interest,  inclu¬ 
ding  the  Gaussian  density,  satisfy  (6).  However, 
one  that  does  not  meet  the  criterion  is  the  La¬ 
place  density  [12] 


By  taking  the  Laplace  density  as  the  uniform  limit 
of  a  sequence  of  densities  (a  -►  0) 

f  .la  1*1' 

fa(x)  «  1/2  e‘lxl[l  -  ote  a  J  (7) 

Fleischer  showed  that  it  also  possesses  a  unique 
minimum  mean  square  distortion  quantizer.  One 


might  then  wonder  about  the  strictness  of  Fleischer's 
condition  (6).  Can  it,  for  example,  be  weakened  to 
hold  almost  everywhere  on  the  real  line?  The  answer 
is  no.  For  example,  the  density  given  in  (4)  satis¬ 
fies  Fleischer's  condition  everywhere  in  [-7,7]  that 
the  second  derivative  exists,  yet  it  has  three 
different  stationary  points  (locally  optimum  quan¬ 
tizers).  By  taking  a  mixture  (l-e)f(x)  + 

ee  //JiT,  where  f(x)  is  given  in  (4),  we  can  get 
a  positive  density  which  satisfies  (6)  almost  every¬ 
where;  but  again,  this  mixture  density  has  three 
distinct  locally  optimum  quantizers.  Clearly,  some 
care  must  be  exercised  in  extrapolating  Theorem  7 
beyond  its  stated  conditions.  The  following  exam¬ 
ples  may  be  illuminating. 

Let  the  cost  function  be  CQ ( t )  =  |t|.  It  is 

well  known  that  for  N=1 ,  the  optimum  output  level 
is  a  median.  Consider  the  sequence  of  densities 


'«<*>  - 


1/2  -  a 


el  sewhere. 


Each  density  has  a  unique  median  and  therefore  the 
unique  minimum  mean  absolute  deviation  1-level 
quantizer  is  Q ( x )  =  0.  As  a  -►  0  the  uniform  limit 
of  the  sequence  is  the  density 

(1/2  1  <  | x|  <  2 

f(x)  * 

(  0  elsewhere. 

It  is  clear  that  this  density  has  infinitely  many 
medians,  and  so  the  optimum  quantizer  cannot  be 
unique. 

A  second  example  illustrates  another  difficulty. 

o 

Let  the  cost  function  be  C0 ( t )  =  t  ,  giving  the  mean 

squared  error  criterion.  In  this  case  the  optimum 
1-level  quantizer  level  is  the  mean.  Consider  the 
sequence  of  densities 


fn(x)  =  (1 


.l)/_L\e-*2/2  +  l/_L\e 

n  ysr/  vW 


•(x-nr/2 


As  n  +  »  the  sequence  converges  uniformly  to  the 
standard  Gaussian  density 

f(x)  -  J-e-x2'2  . 


In  fact,  |fn(x)  -  f(x)I  <  1/n  uniformly.  Neverthe¬ 
less  each  fn(x)  has  a  mean  (optimum  output  level) 

of  1,  whereas  it  is  well  known  that  the  optimum  1- 
level  Gaussian  quantizer  has  the  output  level  at  0. 
So  the  limiting  quantizer  is  not  even  locally  op¬ 
timum. 

These  examples  illustrate  that  uniform  conver¬ 
gence  of  the  sequence  of  densities  Involved  Is  not 
enough  to  assure  that  the  limiting  quantizer  is 
either  optimal  or  unique.  Some  other  ingredients 
are  necessary,  and  these  are  given  In  the  following 
theorem  and  corollary.  This  theorem  is  stated  for 
multivariate  distributions,  but  the  application  to 
univariate  densities  is  straightforward. 

Theorem  8.  Assume  that  CQ(t)  is  non-negative,  non- 
decreasing  for  t  ^  0,  and  continuous.  Let  F  be  a 


-  5  - 


probability  distribution  function  which  has  isola¬ 
ted  locally  optimum  quantizers  Qj.  Suppose  that 

{F^  is  any  sequence  of  distributions  converging 

weakly  to  F,  such  that  C(x-y)  is  uniformly  integra¬ 
te  with  respect  to  {Fr} .  Then  for  each  there 

exists  a  sequence  of  quantizers  {Q;J}  so  that  Q  J 

n  .  n 

Is  a  locally  optimum  quantizer  for  F  ,  and  QJ+Q. 
as  n  -*  op.  n  n  J 

Proof:  Due  to  the  assumption  of  nearest 
neighbor  assignment,  we  can  represent  Qj  by  a 

product  vector  y^  =  {y^ »y2,. • . »yN> »  where  the  out¬ 
put  levels  y,.  e  Rk  are  indexed  in  order  of  in¬ 
creasing  norm.  Since  the  Qj  are  isolated  locally 

optimum  quantizers,  there  exists,  for  each  j,  an 
£0j  and  a  closed  ball 

f  ojWj)  ■  IS  .  IIWjIUCoj) 


at  least  m  local  minima.  Now  if  each  of  the  f  1  s 

n 

has  a  unique  stationary  point,  it  must  be  true  that 
f  has  exactly  one  stationary  point.  This  gives  the 
following  corollary. 

Corollary  9.  Assume  that  CQ(t)  is  non-negative, 
non-decreasing  for  t  ;>  0,  and  continuous.  Let  fn,  f 
be  density  functions  such  that  f  +f  and  C ( x-y ) f n ( x ) 

is  uniformly  integrable  for  all  y  (with  respect  to 
Lebesgue  measure).  Suppose  that  each  of  the  f  's 

has  a  unique  optimum  quantizer.  Then  either  f  has  a 
unique  optimum  quantizer,  or  it  has  several  minima 
which  are  not  separated. 

Going  back  to  the  previous  examples,  we  can  see 
that  in  (8)  the  local  minima  (medians)  of  the  limi¬ 
ting  density  are  not  isolated.  In  (9),  the  uniform 
integrabil ity  assumption  is  not  satisfied.  Needless 
to  say,  both  of  these  requirements  are  met  by  Fleis¬ 
cher's  construction  (7). 

IV.  Conclusion 


in  which  y^  is  the  only  minimum  of  D(Q,F).  Hence¬ 
forth  j  will  be  a  fixed  integer  and  we  may  let 
eQ  *  .  We  will  prove  the  following  proposition: 

For  all  0  <e<eA*  there  exists  an  integer  n 
o  o 

such  that  for  n  >  nQ,  each  of  the  D(Q,Fn)  have  a 
local  minimum  point  inside  the  open  ball 

Ne(Qj)  =  {y  6  RkN:  ||Mj||  <  c) 


centered  on  yj. 

The  proof  Is  by  contradiction.  Suppose  the 
proposition  were  not  true  for  some  positive  e  <  e  . 


Then  we  could  find  a  subsequence  {Fn  }  and  quan- 

nk 

tizers  Qk  so  that  Qk  is  on  the  boundary  of  Ne(Qj) 


D(Qk,F^)  <  D(Q,F  )  for  Q  e  N£ ( Q j ) . 


We  have  studied  the  questions  of  existence, 
uniqueness  and  symmetry  of  optimal  quantizers  with 
difference-based  distortion  measures.  Two  theorems 
are  proved  which  guarantee  existence  of  an  optimal 
quantizer:  the  first  a  necessary  and  sufficient 
condition  on  the  form  of  the  cost  function  and  the 
second  a  sufficient  condition  on  the  distribution. 
Then  we  investigated  a  way  to  prove  uniqueness  of  a 
local  optimum  by  considering  sequences  of  densities. 
Several  examples  are  presented  to  clarify  the  dis¬ 
cussion. 

As  a  final  comment,  we  note  that  the  existence 
results  in  Theorems  3  and  4  can  be  straightforwardly 
extended  to  scalar  uniform  step  size  quantizers,  or 
to  scalar  symmetric  quantizers. 
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This  is  so  because  the  distortion  D(Q,F  )  is  a 

nk 

continuous  function  of  the  output  levels  (Theorem 
1);  so  that  if  it  has  no  minimum  inside  N  (Q.), 

k  ^  J 

then  the  minimum  (corresponding  to  Q  )  must  occur 

on  the  boundary.  The  sequence  (Q  }  has  a  limit 
quantizer  Q"  on  the  boundary,  and  a  slight  modifi¬ 
cation  of  the  proof  of  Theorem  5  yields  that  CP  is 
optimal  for  F  among  the  quantizers  restricted  to 
TV  (Q,).  But  this  contradicts  the  choice  of  c  , 
eo  J  0 

which  proves  the  proposition. 

By  taking  a  sequence  em+o,  we  conclude  that 

every  subsequence  of  {FR}  contains  a  further  sub¬ 
sequence  F  ,  for  which  there  Is  a  sequence  {Qjb 

1  m  1  1  m 

so  that  (r  (r  and  each  Q*  Is  locally  optimum 

nm  nm 

for  F  .  Therefore  the  conclusion  of  the  theorem 
nm 

must  be  true  for  the  entire  sequence  (Fn). 

According  to  the  theorem,  If  f  +  f  and  the 

density  f  has  m  distinct  locally  optimum  quanti¬ 
zers,  then  every  f  with  n  sufficiently  large  has 
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